Large-scale Weakly Supervised Learning for Road Extraction from Satellite Imagery

Automatic road extraction from satellite imagery using deep learning is a viable alternative to traditional manual mapping. Therefore it has received considerable attention recently. However, most of the existing methods are supervised and require pixel-level labeling, which is tedious and error-prone. To make matters worse, the earth has a diverse range of terrain, vegetation, and man-made objects. It is well known that models trained in one area generalize poorly to other areas. Various shooting conditions such as light and angel, as well as different image processing techniques further complicate the issue. It is impractical to develop training data to cover all image styles. This paper proposes to leverage OpenStreetMap road data as weak labels and large scale satellite imagery to pre-train semantic segmentation models. Our extensive experimental results show that the prediction accuracy increases with the amount of the weakly labeled data, as well as the road density in the areas chosen for training. Using as much as 100 times more data than the widely used DeepGlobe road dataset, our model with the D-LinkNet architecture and the ResNet-50 backbone exceeds the top performer of the current DeepGlobe leaderboard. Furthermore, due to large-scale pre-training, our model generalizes much better than those trained with only the curated datasets, implying great application potential

Finding the Pillars of Strength for Multi-Head Attention

Recent studies have revealed some issues of Multi-Head Attention (MHA), e.g., redundancy and over-parameterization. Specifically, the heads of MHA were originally designed to attend to information from different representation subspaces, whereas prior studies found that some attention heads likely learn similar features and can be pruned without harming performance. Inspired by the minimum-redundancy feature selection, we assume that focusing on the most representative and distinctive features with minimum resources can mitigate the above issues and lead to more effective and efficient MHAs. In particular, we propose Grouped Head Attention, trained with a self-supervised group constraint that group attention heads, where each group focuses on an essential but distinctive feature subset. We additionally propose a Voting-to-Stay procedure to remove redundant heads, thus achieving a transformer with lighter weights. Moreover, our method achieves significant performance gains on three well-established tasks while considerably compressing parameters.Comment: In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 2023

Logical Reasoning over Natural Language as Knowledge Representation: A Survey

Logical reasoning is central to human cognition and intelligence. It includes deductive, inductive, and abductive reasoning. Past research of logical reasoning within AI uses formal language as knowledge representation and symbolic reasoners. However, reasoning with formal language has proved challenging (e.g., brittleness and knowledge-acquisition bottleneck). This paper provides a comprehensive overview on a new paradigm of logical reasoning, which uses natural language as knowledge representation and pretrained language models as reasoners, including philosophical definition and categorization of logical reasoning, advantages of the new paradigm, benchmarks and methods, challenges of the new paradigm, possible future directions, and relation to related NLP fields. This new paradigm is promising since it not only alleviates many challenges of formal representation but also has advantages over end-to-end neural methods. This survey focus on transformer-based LLMs explicitly working on deductive, inductive, and abductive reasoning over English representation

Reduced scale model test on cable membrane roof of Shangai Expo Central Axis

p. 2008-2018In this paper a reduced scale model test on cable membrane roof of Shanghai Expo Central Axis is introduced. The membrane pre-stresses, cable forces and membrane geometry at the initial state are carefully inspected. Numerical form-finding analysis is also carried out and its result is compared with the inspecton. The behaviors of the membrane roof under breaking of cables are observed. Test proves the practicability of the project in aspects of system safety, analysis and inspection.Zhang, Q.; Yang, Z.; Chen, L.; Tang, H.; Zhu, B. (2010). Reduced scale model test on cable membrane roof of Shangai Expo Central Axis. Editorial Universitat Politècnica de València. http://hdl.handle.net/10251/718

Large Language Models for Automated Open-domain Scientific Hypotheses Discovery

Hypothetical induction is recognized as the main reasoning type when scientists make observations about the world and try to propose hypotheses to explain those observations. Past research on hypothetical induction has a limited setting that (1) the observation annotations of the dataset are not raw web corpus but are manually selected sentences (resulting in a close-domain setting); and (2) the ground truth hypotheses annotations are mostly commonsense knowledge, making the task less challenging. In this work, we propose the first NLP dataset for social science academic hypotheses discovery, consisting of 50 recent papers published in top social science journals. Raw web corpora that are necessary for developing hypotheses in the published papers are also collected in the dataset, with the final goal of creating a system that automatically generates valid, novel, and helpful (to human researchers) hypotheses, given only a pile of raw web corpora. The new dataset can tackle the previous problems because it requires to (1) use raw web corpora as observations; and (2) propose hypotheses even new to humanity. A multi-module framework is developed for the task, as well as three different feedback mechanisms that empirically show performance gain over the base framework. Finally, our framework exhibits high performance in terms of both GPT-4 based evaluation and social science expert evaluation

A Survey on Semantic Processing Techniques

Semantic processing is a fundamental research domain in computational linguistics. In the era of powerful pre-trained language models and large language models, the advancement of research in this domain appears to be decelerating. However, the study of semantics is multi-dimensional in linguistics. The research depth and breadth of computational semantic processing can be largely improved with new technologies. In this survey, we analyzed five semantic processing tasks, e.g., word sense disambiguation, anaphora resolution, named entity recognition, concept extraction, and subjectivity detection. We study relevant theoretical research in these fields, advanced methods, and downstream applications. We connect the surveyed tasks with downstream applications because this may inspire future scholars to fuse these low-level semantic processing tasks with high-level natural language processing tasks. The review of theoretical research may also inspire new tasks and technologies in the semantic processing domain. Finally, we compare the different semantic processing techniques and summarize their technical trends, application trends, and future directions.Comment: Published at Information Fusion, Volume 101, 2024, 101988, ISSN 1566-2535. The equal contribution mark is missed in the published version due to the publication policies. Please contact Prof. Erik Cambria for detail

Eau

Modeling aluminum (Al) particle-air detonation is extremely difficult because the combustion is shock-induced, and there are multi-phase heat release and transfer in supersonic flows. Existing models typically use simplified combustion to reproduce the detonation velocity, which introduces many unresolved problems. The hybrid combustion model, coupling both the diffused- and kinetics-controlled combustion, is proposed recently, and then improved to include the effects of realistic heat capacities dependent on the particle temperature. In the present study, 2D cellular Al particle-air detonations are simulated with the realistic heat capacity model and its effects on the detonation featured parameters, such as the detonation velocity and cell width, are analyzed. Numerical results show that cell width increases as particle diameter increases, similarly to the trend observed with the original model, but the cell width is underestimated without using the realistic heat capacities. Further analysis is performed by averaging the 2D cellular detonations to quasi-1D, demonstrating that the length scale of quasi-1D detonation is larger than that of truly 1D model, similar to gaseous detonations

Language Models as Inductive Reasoners

Inductive reasoning is a core component of human intelligence. In the past research of inductive reasoning within computer science, formal language is used as representations of knowledge (facts and rules, more specifically). However, formal language can cause systematic problems for inductive reasoning such as disability of handling raw input such as natural language, sensitiveness to mislabeled data, and incapacity to handle ambiguous input. To this end, we propose a new paradigm (task) for inductive reasoning, which is to induce natural language rules from natural language facts, and create a dataset termed DEER containing 1.2k rule-fact pairs for the task, where rules and facts are written in natural language. New automatic metrics are also proposed and analysed for the evaluation of this task. With DEER, we investigate a modern approach for inductive reasoning where we use natural language as representation for knowledge instead of formal language and use pretrained language models as ''reasoners''. Moreover, we provide the first and comprehensive analysis of how well pretrained language models can induce natural language rules from natural language facts. We also propose a new framework drawing insights from philosophy literature for this task, which we show in the experiment section that surpasses baselines in both automatic and human evaluations. We discuss about our future perspectives for inductive reasoning in Section 7. Dataset and code are available at https://github.com/ZonglinY/Inductive_Reasoning.Comment: Accepted by EACL 202

Initiation structure of oblique detonation waves behind conical shocks

The understanding of oblique detonation dynamics has both inherent basic research value for high-speed compressible reacting flow and propulsion application in hypersonic aerospace systems. In this study, the oblique detonation structures formed by semi-infinite cones are investigated numerically by solving the unsteady, two-dimensional axisymmetric Euler equations with a one-step irreversible Arrhenius reaction model. The present simulation results show that a novel wave structure, featured by two distinct points where there is close-coupling between the shock and combustion front, is depicted when either the cone angle or incident Mach number is reduced. This structure is analyzed by examining the variation of the reaction length scale and comparing the flow field with that of planar, wedge-induced oblique detonations. Further simulations are performed to study the effects of chemical length scale and activation energy, which are both found to influence the formation of this novel structure. The initiation mechanism behind the conical shock is discussed to investigate the interplay between the effect of the Taylor-Maccoll flow, front curvature, and energy releases from the chemical reaction in conical oblique detonations. The observed flow fields are interpreted by means of the energetic limit as in the critical regime for initiation of detonation